Website development systems are designed for developers, third party application developers and end users to design, develop and market their own tailored websites. There is a constant need to update the website system in question and often it may be necessary to determine whether changes are feasible before they are implemented.
Some changes may be specific to particular geographic populations (such as all users located in a particular country), technology specific (all iPad users) or may be specific to a particular system feature etc. Such changes often start from a hypothesis that the change (or new feature) would have a positive effect on a key performance indicator—the success metric. Thus it is often desirable to release the updated feature to the pertinent population, without their knowledge and to follow up and monitor its use based on pre-defined success metrics. The results of such experimental release would determine whether the change (or new feature) is successful as hypothesized or not.
Experiment management systems are known in the art and provide the opportunity to accelerate innovation by evaluating ideas quickly and accurately usually using controlled experiments such as A/B and multivariate tests with the underlying system. In these tests, a change is usually measured against a baseline version of the system.